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RESEARCH ALLIANCE FOR NEW YORK CITY SCHOOLS 


The Research Alliance for New York City Schools 
has worked with the New York City Department 
of Education (DOE) and other key stakeholders 
since 2008 to “advance equity and excellence in 
education by providing non-partisan evidence 
about policies and practices that promote students’ 
development and academic success.” 1 Per a leader 
of the Research Alliance, its goal is “to conduct 
rigorous studies of questions that matter to 
policymakers, practitioners, and other stakeholders 
in New York City schools." 

Housed in New York University’s (NYU) Steinhardt 
School of Culture, Education, and Human 
Development, the Research Alliance’s work is 
guided by a governance board that represents the 
DOE (the Chancellor of New York City Schools), 

NYU (the provost), and representatives from 
district stakeholders (e.g., unions) and community- 
based organizations. This governing board sets 
the research agenda, which, in recent years, 
has focused on four key issues: (1) high school 
achievement, attainment, and post-secondary 
preparation; (2) achievement and development 
in the middle grades; (3) contexts that support 
effective teaching; and (4) data use for practice and 
policy. 

Currently, the Research Alliance is staffed by an 
executive director, three full-time researchers, and 
several research associates and programmers. 
Research Alliance staff work primarily with the 
leadership of the Office of Research, Accountability, 
and Data at the DOE, with whom they meet 
monthly. They also work with staff in other 
departments at DOE when a given study addresses 
issues under their purview. 

History 

After years of conversations and convening with 
a range of interested parties in New York City, the 
idea of an independent research organization to 
provide nonpartisan information about New York 
City Schools began to come into view in the 2006- 
2007 school year. That year, New York University 
professor Richard Arum devoted his sabbatical 
year to creating such an organization. He began by 
creating a series of advisory groups with a range 
of stakeholders (area researchers, representatives 
from the DOE, unions, and community-based 
organizations) to focus on laying the groundwork 
for different aspects of the organization. Arum 


then organized a conference to discuss a series 
of commissioned papers on different approaches 
to research partnerships and on the types of 
issues relevant to New York City schools. Finally, 
he convened a governance board made up of key 
leaders in New York, including William Bowen 
(then president of the Mellon Foundation, and a 
former president of Princeton University), Randi 
Weingarten (head of the American Federation 
of Teachers, and the former head of the New 
York City branch of the American Federation of 
Teachers), Joel Klein (then Chancellor of New 
York City Schools), the head of the New York City 
Chamber of Commerce, and directors of a number 
of community-based organizations and school 
reform organizations. The governance board was 
charged with focusing on the critical ingredients 
necessary to develop an independent research 
organization. They identified three tasks necessary 
for developing the independent research group: 

(1) identifying a sponsoring organization, 

(2) raising start-up funds, and (3) hiring an 
executive director. 

The board was able to accomplish all three tasks. 

It secured a commitment from NYU to act as the 
sponsoring organization; was awarded a large 
start-up grant from the Bill & Melinda Cates 
Foundation, as well as smaller grants from 1 1 other 
foundations; and hired an executive director. In the 
four years since its inception, the Research Alliance 
has secured additional general operating support 
from the Robertson Foundation and more grants to 
fund specific research projects. 

Nature of the Partnership 

The Research Alliance works with the district in 
three main ways. First, it conducts evaluation 
studies on policy and practice issues associated 
with the research agenda set by the governance 
board. Recent studies have included an 
investigation of teachers’ use of New York City’s 
Achievement Reporting and Innovation System 
(ARIS), an examination of students’ transitions 
in and through middle school, and a look at the 
impact of the planned closure of underperforming 
schools on the achievement of students attending 
those schools. Second, the Research Alliance has 
developed a longitudinal data archive, thanks to 
data-sharing agreements with the DOE that give 
it access to a broad range of district data. Third, 
the Research Alliance has initiated “descriptive 
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and formative” work, which involves formative 
research in the service of capacity-building. 

An example of this third line of work is a 
collaboration between researchers and DOE 
administrators to improve the measures of 
school environment on the DOE’s annual survey. 
According to district administrators, the survey 
is the second largest census-style survey in the 
country — second only to the U.S. Census. It is 
administered to nearly 2 million teachers, parents, 
and students in grades 6-12 at more than 1,700 
schools. The DOE typically receives more than 
900,000 responses, a 53 percent response rate. 

The survey plays an important role in New York 
City schools. Staff at the school- and district-level 
use the survey results to inform their improvement 
work. In addition, measures from the survey, along 
with attendance figures, comprise up to 1 0-1 5 
percent of the grades that schools get on their 
annual school progress reports. The grades, if low, 
can result in intensive support and intervention. 
High grades can result in bonuses for school 
principals. One DOE administrator explained: 

The survey is not just an informational 
or research-driven survey. It has 
implications. One, in terms of being 
involved in the progress report, but two, 
also, it’s hopefully seen as a treasure 
trove of data for a school in terms of 
what their constituencies are getting or 
not getting. 

The survey collaboration began when the Research 
Alliance initiated an analysis of the survey 
measures related to school learning environment. 

It embarked on this work because of its potential 
to have system-wide impact and because it met 
some research goals for non-academic outcomes 
and learning environments. At the same time, 
district administrators were getting requests 
from people in schools for more “actionable” 
information from the survey. The administrators 
saw an opportunity to work with the Research 
Alliance to update the survey, making it more 
useful to schools and improving its technical 
quality by better differentiating across schools. 

Research Alliance staff first analyzed the validity 
and reliability of the school environment questions 
on the teacher survey. The researchers discovered 
that although the district had intended to measure 


several distinct aspects of the school environment, 
the results in different categories of measures 
were highly correlated and were likely only 
measuring more global qualities. The measures 
also tended to find differences within, rather 
than between, schools. Thus, the survey did not 
distinguish between schools with strong versus 
weak learning environments. 

Based on the analysis, the Research Alliance, with 
approval from the DOE, gathered and evaluated 
survey measures from other sources that had 
well-established validity and reliability and did a 
better job distinguishing between schools. After 
considerable discussion with a panel of survey and 
measurement experts (which the Research Alliance 
convened), the DOE selected new survey items, 
which were piloted on the 2012 teacher survey. 
The Research Alliance is now engaged in an 
analysis of the results of the new items. If they are 
an improvement over the existing items, they may 
become part of the measures used for the school 
progress report. The DOE has asked the Research 
Alliance to embark on a similar process with the 
student and parent surveys. 

Challenges 

There are, of course, challenges involved in 
developing a strong partnership. Individuals from 
the Research Alliance and the DOE identified three 
they had faced. First, research alliances involve 
new and different roles and relationships between 
researchers and district leaders. The researchers 
are not consultants. The district has not hired 
them, but they are not fully independent, either. 
The success of the work depends upon the ability 
to maintain relationships over a long period of 
time. One researcher described the situation: 

We’re not consultants who have a 
contract. We’re not a branch of the DOE 
that has a vested interest in promoting 
or defending its policies and programs. 
There’s constant negotiation without 
clear parameters and ultimately, both of 
our organizations’ priorities need to be 
met. 

Second, researchers are caught between being 
responsive to district timelines and maintaining 
their commitment to high-quality research designs 
and analytic techniques. As the relationship 
between the Research Alliance and the DOE has 
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matured, the DOE has increasingly requested 
assistance and additional analyses, often with 
a quick deadline. This has challenged Research 
Alliance staff to find ways to be responsive, 
while maintaining careful, deep analysis. As one 
researcher explained: “This is the danger ... the 
rush of what we need to use right now for this 
particular project. The price is the more formal 
research paper.” 

Third, it can be challenging for the district to be 
responsive to research findings, especially in large 
urban districts in which decision-makers must take 
multiple factors into account in policymaking . 2 
Findings produced by the Research Alliance have 
not always been used to craft DOE policy. A DOE 
administrator explained: 

My team always tries to connect the 
Research Alliance researchers who are 
working on a particular subject area to 
the offices within the department that 
own that area. [The response] varies. 

Frankly, it depends on the type of report 
that they’re doing, or the type of project. 

If it’s a retrospective research report, 

I think they’re always very interested 
to hear the findings and will file them 
away. 

As other research alliances have discovered, it may 
not be enough to produce high-quality research. 

You may also have to create systems and structures 
to support implementation and follow-through . 3 
DOE administrators point to the survey work as a 
model for Research Alliance engagement in both 
research and implementation. 

Benefits 

The Research Alliance is a relatively new 
organization, and it may take some time for a 
full portrait of the benefits of the partnership 
to emerge. Individuals on both sides of the 
partnership, though, already identify several 
positives. The partnership has clearly resulted in 
improvements to the teacher survey. Given that the 
survey has “real-world consequences” related to 
accountability and decision-making, the changes 
to the survey can potentially influence policy and 
decision-making. 

The process of working together to improve the 
survey has brought new resources and capacity 


to the DOE. DOE administrators note that the 
Research Alliance’s efforts to find and evaluate 
questions from surveys across the country and its 
ability to convene experts to review them brought 
new ideas, expertise, and resources into the 
district. One administrator noted: 

It was great to have partners with a lot 
of preexisting knowledge about the other 
surveys being used around the country, 
who also had the time and the expertise 
to dig up new examples, and develop 
new models that we could use. We don't 
have that expertise internally, and we 
don't have the time to do the extensive 
searching. 

The partnership has also built DOE administrators’ 
capacity to do this work and led to the production 
of high-quality research on New York City Schools. 

The long-term and sustained nature of the 
partnership between the Research Alliance and 
DOE has lowered start-up costs. Repeated contacts 
across a range of different projects have led to 
institutional agreements and relationships among 
partnership members. For example, Research 
Alliance and the DOE do not need to draft new 
“Memos of Understanding” or data-sharing 
agreements for each new project, because they are 
already in place. Working together over time also 
fosters the development of trust that enables freer 
and more productive exchange of ideas. Because 
the collaboration is long-term, the Research 
Alliance has been able to develop a sophisticated 
history and knowledge of NYC public schools. A 
district administrator explained: 

We do have a lot of researchers who 
make use of our data, want to conduct 
studies in our schools, or get brought on 
to do evaluations. What’s really unigue 
about the [Research Alliance] is that it’s 
a sustained relationship across multiple 
projects, over a long period of time. ... 
Having a thought partner about our 
data is really important. 


For more information about the Research Alliance 
for New York City Schools, see: http://steinhardt. 
nyu.edu/research_alliance 
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